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ABSTRACT 



Models of five alternative ways to; approach the 
evaluation of organizational develppment (OD) pro jects' are sketched : 
xTI) ask the expert; (,2) w^hat's the target; (3) did ve hit the^arget; 
(^) mid-course correction; and (5) continuous monitorilig. • Each model 
provides a different perspective on the problem of evaluation and 
each contains certain advantages and disadvantages. The five 
approaches are next put into organizational^, context Jby a survey of 
some major obstacles to effective evaluation studies. Persistent ' 
obstacles, ^are found in the client^ in the OD consultant^ and within 
the OD actiVitv itself . In the final s^ection^ two issues are 
addressed that mqist be , f aqed in any effort to evaluate an iOD 
program — clarity of the role of eval^uation and the audience of the 
evaluation. (Author) 
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'How are we doing?"' is a question often asked amid the uncertainties > 
ambiguities, anxieties, /and newness that accompany many attempts at improv- 
ing organizationaJL effectiveness. , While thci people involved in these 
activities are raisinjg^ this question, their supervisors and other interested 
onlookers, on the put$ide are raising the question of ^ accountability . Those 
supplying reso^ces/or those onlooiters who see themselves bi(»ing affected by 



any 'outcomes /6f an OD project are likely to be interested Ijn measuring the ' 

/ \ ^ \( - ■ 

product-or evaluating the outcomes. Their question , *Vhat are you 

r 

doing?" S6, two -questions are frequently raised, often by different parties 
carrying^ different perspectives: "HoW are we doing?" and 'Vhat are ^6u 
doing?" This* paper provides some ways of thinking about these questions 
and some alternative models for evaluating organization developmeni^ efforts. 

( ■ 

Our interest in writing this paper stems from i^ree conci^rns or v 
beliefs. VThe fiirst is the frustrati<|i we have experienced in trying to in- 
duce organization]? to devote time, money, emd other resources to evaluating 
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I ' ■' ' '•/ ' 
OD efforts/ The secon^ is our b)^lief that evSluAtlon: has 1?bo often been 

viewed in a non-dif'fe^ientiatctd/ all-pr-nothing vay. We reject the idea 

that there is one be^t way to (Evaluate organization |levelopraent . '*We^see 



instead several kin^s of apprpjriate evaluAtional activities rane^iAg on' 
a continuum from careful documentation of [what occurs in an OD project, 
through process feedback, Vo/ evBluation of outcomes, and finally to field 
research where specific hyj^itheses or alternative "approaches are rigorously 
tested using the skills ana safeguards of the research scientist* Ml 
thes^ ajyproachips address tinemselves to the questions, "How are we doing?" 
and* "What are you doing? "i 

This- paper does not /focus on how to conduct evaluation sti^ies. 

/ - . • "\ 

Hopefully, it lays out considerations and raises issues vl^hich will enable , 

the reader to make bettw decisions about how his own OD projects can most 
effectively be ev^i^uated. ' 

We begin by offeri^ig five al^^emative approaches to e^raluation and 

give advantarges and disadvantages of each approach. Next; \these five models 

\ 

a:^e put into an organizational context by a survey of some *pf the persistent 
obstacles to effective evaluation. Finally, two major issues in the design 
and execution of any evaluation sche|ne are outlined. 

We hope the reader becomes more aware of the alternatives available to 

.him in evaluating OD activities, and recognizes that there is not a single 
best approach to evaluation. We also hope to increase appreciation for the 
barriers that exist to evgiluation and to Increasei understanding about why 
they exist. In our judgmeTrt", people hkve frequently deplored the ]^ck of 
evaluation studies and have tried to cajole one anbther into, doing more 
and better evaluation of OD projects without a thoibu^^h understanding (>% 

' the reasons for lack of a history of sounci .studies iWaluating or^izatlon 
development. 



Five Models 




Let's begin with a committee working otx a task. How does a coinmittee 
know hoiir it'k doing on /its task and how do ihe members/ know how they are 
performing as parts op the problem-solving process? iSow does the organiza- 



\ 



ti 



know what cont^ributi>^nJbhe committee mak^s to thjie goals of the organi- 



"Zatf^^ There ar^ s 

Olid its members. Each provides us. with a different model by which we can 

approach theNfivaluation of organizatljon development projects, 

- ' \^ . '% » ' 

Here are five models: Ask the expert; {P) V/hat's the target?; 

^ - . 

(3) Did we hit the target?; (U) Mldrco^rse^ correct^ j on ; and Cont'lnuoun 
monitoring. Each of th.ese appr^oaches provides a different porspectivo and 
each carries certain advantages and dlaadvanta^^s (Weiso & Rein, 1^70). . 

1, Ask the expert . A consultant is asked to provide an independent 
evaluation of etn .OD program. The question i8,^^"Tell us how we're doing or 
how they're doi;ng!" Th^ answer Is giv.en by an e'^ert. This model usually 
calls' for evaluation following the completion of OD activities, but, expert 
assistance can take place at any point In the life of a developmental pro- 
ject. While expert evaluation is often given by /m outside consultant In 

the form oC a research contract, an internal consultant can alflo offer an 

> 

expert evaluation. However, the -closer the ©valuator in ^to the organization 
and its project, the more likely he is to take the project's basic assump-- 
tions and organizational arrangements as given and conduct his^evaluation 
within the existing framework. An outsider may be able, on the other hand, 
to exercise more autonomy in ^ his questioning aijd take an independent per- 
spective simply because he i^ not part of the cul^ture. **The iinplicationB he 
draws from evaluation data may be oriented less to tinkering and more to 
> fundamental restructuring of the program" (Welaa, 1972a, p. 2l). 



veral ways t'o evaluate the i>erf|)rmance of a committee 
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One dilemma in ail^g the expert is that the crecllbility of his re- 
sponse may differ (|epepding on the audience, 'Those outside the OD acti- 
vity, who' are askirig for accountability, are less likely to view hia * , 
assessment as biased or self-serving than if the same information wer^to 
come from 3>eople involved in the chanp^e process. Whereas for the later, 
unless'the ex]|^rt is able to establish, that ^e has eui empathic und«r- 

standing of the people who are participating" in the change process, ^^i"B — ^ 
<» 

I X ^ . " 

credibility with them may be low or suspect. / . X' 

Advantages r ^ ^ ^ 

(a) The uae of experts helps us avoid continuously reinventing 
the wheel* An tbq^ert provides ^orapetenc? in research design,. 

^data collection Aand data ano^lysis techniques. He alao pre- 
• parcTs .aind presents reports that are both technically sound 
• » ^ and readable . 

• ^ : ' V • 

•(b). When the evaluation is assigned to an uninvolved expert, it 
is not likely-to get lost or compromised as the "suction" of 
the OD project increases, - - * 

' (c;) Where evaluation is conducted by an expert, the comparability 
J:"" of studies is greater, and therefore , J u^t as expert evaljua- 

tion b^llSLs on prior learning it also adds to the accumula- 
tion *of an Ot) literature which may be useful to others. 

Disadvantages : 

; (a) Relying on an expert may foster dependence on the competencies 

. of othe?lrs . Internal resources are then not developed with 
^ competencies to monitor their own projects* 

(b) While an outside expert is likely to dome with a theory or 

^ model, to guide his evaluation, and this is often on advantage, 

because of his lack of involvement in the particular OD enter- 
prise, he- may iapose an inappropriate model for evaluation, 

(c) The expert's evaluation may be out of step with the needs of 
the change process itself. The expert's report is often too 
la/te to become incorporated v^ithin the OD activity. There is ^ 
difficulty in synq^pni^ing both the collection of relevant 
data and the expej:^^ feedback with the progress of the OD 
project itself (WeisVl972b)^. ^ 

(dTj It is the expert, not jthe client organization, that "oima" 
the evaluation data. rThis magnifies the problem of accept- 
ance of the data. 



(e*). What if you rely on an expert, aind he Is not really an 
expert at all? Come people are experts in organization 
' development" without understajiding evaluation research 
very well. . 

2. Wiiat's the target ? Just as the progress of a conimlttee meeting can 
be compared against a clearly spei^ifled and agreed upon agenda, so an OD 
effort can, be compared with what it^ started out to . accomplish . The process 
of j^tablishing a target is part of what is often called, "clarifying the 
contract."- Is "the target (or targets) clear? Are we agreed on our objectives? 
Are we all Tleaded the same direction? If there are multiple objectives, 
do they complemen/ ^one another, are they independent of one another, or are 
they antithetical fi^^ one another? ^ 

Sometimes objefcjtlves , roles, and procedures are established because they. ^ 
appear both treasonable and necessary for approval of an OD project by top 
maJiagement or funaing sources or granting agencies. Yet, once the project 
is launched these ^'public" obJ^ectiVes may seera tinrealistic and inappropriate 
to the new situation. An organization continually changes, as do tiine frajnes , 
priorities, urgencies, personnel, etc. Therefore, a useful first step is the 
translation of these puWLic objectives into f obtainable and measurable targets. 

Advantages : ^ 

(a) A major goal of OD projects is ciTten to achieve clarity in 
problem-def 5nition and goal-setting, and then to linX up 
organizational resources toward achieving these agreed 
upon objectives. To establish- a clear and agreed upon 

V target therefore represents an accomplishment in itself 

(Beck & Hiljnar, 1972-). 

(b) The earlier; evaluatioi^ is agreed upon, the- lees likely 
evaluation will be loat in ensuing activities. 

Disadvantages : 

(a) Where there is a press for early convergence, this may 
lead to conformity, and thereby to solutions which are 
satisficing rather than optimizing. 
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(b) ' While clear and' early attention td goals represents in J - 
itself a contribution to organizational effectiveness, / 
action must fpllovr otherwise people rriay experience con- 
fusion between stating, objlectives and obtaining results. 

3. Did we hit the (target ?. It is sometimes thpdght that the progress ^ 
of a meeting can be side-tracked by evaluating it|? effectiveness while it ' 
is still in progress. It is also sometimes thouglit that organizational 
qhange is best measured Sstfter the cheiaige effort has bpen completed. Thus, 
some evaluation schemes urge,- "wait until we're finished and then we'll see 
if we hit cmr target." In contrast to the second approach, this strategy 
assumes tile objectives, resources, and the people involved as given, not 
as factors which themselv<^ are sub,iect to change. The question is, there- 
fore, "Given this combination of objectives, procedures, resPurc.es, and 
peopl^, what did we accomplish?" 

Advantages : ' * . 

(a) : By focusing on the extent to which the deaired outcomes 

have been achieved, as opposed to examining the means used 
.to reach the tapget, people are more free to choose their 
"own in^ tin ^omental' actions to reach thel^ objectives. 

(b) Just as mar^tagers often appear to be "process blind," so ^ 
Ol) consultants are sometimes "task blind." It is on^sy 

to become so enamored with the dynamics of process , tjiat' 
\^one'3 evaluation of an OD effort ma> become sidetracked in 
considering process variables at the expense of end results 

(c^) The evaluation of an end product meets the needs of onlook^s 
or others concerned with acc6untability . 

^ Disadvantages : 

^(a) Thig evaluation assumes that the target is the moat important 
thing to evaJ,uate. I{ad this perspective been applied in the 
, Hawthorne studies., we might have learned that illumination 
levels are, in fact, associated with higher productivity 
^ (Roethlisberger h Dicksot^, 1939).. \ 

(b)^ At the termination of a project, energy is likely to be 

diffuse and interest turned elserwhere, so evaluation is often 
difficult. 
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(c) Tf the target Is achieved and that's all one learns, the 
kej[^eatiires in understanding how or why the enterprise 
was successful may be iwi'ssed. If the target is missed, 
it may be precisely bjerause earlier opportlmities for • V 
modification were bypassed and the collection of data was ^ — ' 
delayed until 'the project was completed/ 

Bbth of the problems in (c) are greater if an OD event is 
a one-shot occurrence , rather than the first in a sequential 
series of programmed developmental activities. 

Mid-course correction . One item towards the mi^ddle of the agenda 

of a meeting might be, "How are we doing so^r?" At the onset thea^^ is 

agreement to evaluate the progress of the meeting at a predetermined point 

(or points) with the clear intention that the meetinK can be modified 

should the new information call for such action. A similar plan— can be uaed 

to evaluate the^progress of an OD pro.lect. - . 



Advantages : 



|>rc 



(a) "^ It ^provides opportunity for change based "on neyr information 

or new circunstances . An opportunity is also provided to 
assess the original goals . '^hls is particularly appropriate 
y where the project is of long duration or occupies a small 

percentage of people ' s time and attention on th)e Job. 

cleeir expectation is eotabllahed that the focus is to 
shift to evali:ffl!:i:ion at a p:iven point in time, -^hereby avoid- 
ing the seduction of the ta.sk the excluiJion of measuring 
profrress . • * 

"\ 

Disadvantages : 

\ N 

(a')V;irt provides a conflict for participants- Are they 'to be ^ 
"good soldiers" by being loyal and optimistic supporters 
of ,^the proppj'am or can they be open to contrary information. 
' This conflict is particularly^ sharp where onlookers are 

Q\ skeptical. If dubious viewers question the value pf the 

activity In which one, is involved, it is difficult to admit 
' publicly that there a.t^ some real drawbacks to the activity. 

(b) One may ovorreadt to point-in-time information, if, the pro- ^ 
' ject is asseasf^d at the top of an upswing or at the bottom 

of a downturn* 

5. Cont,inuoufl monitoring . An integral and continuoos part of any OD 
effort is the expectation that we'll check on ho^ we're doing by continuously 
generating relevant information. ^At the very heart of the concept of 

--■<)■ 
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organization development is the belief in the organ4zatlon;s capacity to 
generate rej.evant information and its-lability to act on such- Information** 
^ In waysHhat both expand and utilize resource^ effectively (Argyris, 1970), 
Thus, it is easy to see on evaluation process . which continually monitors 
progress as the super-solution, is therefore important to recognize that 
there are limitations and disadvantages to this method, and also to remembpr 
that the other four models are respectable alternatives, each with its own 
strengths. 

Advantages : • 9 * ' ^ 

/ ' ■ • • 

• (a) ' Cycling new information through the organ i^elt ion, to check 

J- progress, expectations, and resources is^ itself ^ntral 
^ to the developmental prOcess . Such an ^valuaticW^ cycle 
^ builds competencies and linkages between acCion--riew informa- 

tion --feedback --act ion . 

(b) Evaluation ii3 most timely if it Is initiated by hew informa- 
tion entering the system, rather than by a prearranged time 
or by the termination of the project. 

(c) Sometimes environmental imperatives require us to begin an 
OD project without a programmatic model of change or a clear 1 
specified target. In this case, a continuoua process <if/evalua 

< tion provides data about the self -consistency of ' our approach 

to problems and also about the emerp:lng p^oals the OD project 

Disadvantages : ' * 

(a) This model calls for a high level of sophistication and skill 
* by all parties involvecl. It requires the ability to move 'In 

highly disciplined and flexible ways from the demands of the 
task to fruitful examinations of^the process by which people 
are working together. In ehoosing ' this model, the difficulties 
in achievinf;^ these skills and developing the required disci- 
pline should not be underestimated. 

(b) The language developed, techniques employed, and the data* 
utilized may become quite idiosyncratic to the parties in- 
volved, and therefore perhaps, less understandable and per- 
suasive to others. It may be difficult for others, to learn 
from this approach to evaluation. 

What is Ifearned from this evaluation model is more likely to 
b^ a set of skills geared to monitoring beha>/ior within the 
system, and is less likely to provide data for accountability 
outside the project. This model is, therefore, probably not 
very^persuasive to' persons who are not themselves involved In 
theNQD^ project . 
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^~ (c) Due to the constancy/of'-the^ evaluation cycle whenever new 

information is available, the focus of attention is probably 
on small units and one may be unable to see "the ^j^^orest for 
th©» trees."' Leamin-gs eLnd insights raay be lost and must-be 
rediscovered in subsequent pro^^ects, because the perspective 
is Limited to short-te rm events*. 

(d) For small task forces this may V)e the model of cheice, 
becaus^ the /smaller the unit the more appropriate and 
y^ossible are fi^quent review and evaluation. Due to the 
constraints of time, location, and competing tasks, it is 

*^ difficult for larfTje units or systems to continuously eva- 
luate their progreKs. 

While all five models have their strengths aml:.t]}^44 itroblems, they all 




suffer .in varying degrees from defining the qii^t^on of evaluation too narrow 
ly- Three of the models focus on point-in-time measures (what's the target; 
did we hit the target; and mid-course corre(?tion) . One of the other two 
models relies heavily on outside evaluation (i.e., ask the expert), wheree^s 
the continuous monitoring model relies heavily on internal resources. To 
provide information to onlookers and those concerned with accountability, 
asking the expert or asking did we hit the target probably perfonn better. 
As contributions to the development of resources internal to the project 
itself, early work on the target, mid-course oorrertior s , und continuous 
monitoring probably offer more advanta^7es. 

Obstacles to Affective Evaluation 

' To many persons actively engaged in 6rganization development, the need 
for more frequent and effective efforts, at evaluation are clear; yet, there ^ 
is widespread embarrassment duetto o»ir persistent ^ck of attention to evalua 
^tion and due to the lack of enduring credibility of our own folklore. Why 
' isn't there^a more sterling record of evaluating the re'sults of OD efforts? 
We see three sources of the problem. There are obstacles to effective \ 
evaluation (a) in th)e client, ('b) in the 01) , practitioner hi*mself or herself, 
as weir as (c) within the OD activity itself. 



10 



Obstacles In the Cl ient - ^ 

As stated at the beplnnlng of this paper the questlons--"How are we 
doing?" and "What are you dolngV*'--are likely to ar>3e In any OD effort. 

if 

since both questions place pressure on tTie client to emluate what he is 
doing, it is strange that there Is such a paucity of well -regarded documien- 
tatlon and research on organization development. ^One way to understand the 
lack of evaluation studies, Is to assume that there must also exist within 
thep^lent strong countervailing forces serving as obstacles to evaluation. 
In Wdition,*to the time and cost roncerns which are Inherent in any decision 
to commit an organization to act:|.on, other more specific forces inclufie (a) 
a-belief t^t" evaluation may disrupt the flow of the OD project; (b) a belief 
that the client system has already made Its evaluation when It decided to 
launch the "'^^j^^lect ; (c) a distru.st that an evaluation study Can adequately 
measure and report the real benefits of the pro.iect ; and (d) a fear of nega- 
tlv^or erabarra'ssinfr data- These four forces are now discussed in more detail. 

' Evaluation as disrupfel^ . In most organizations research and devolorpment 
activities are typically- Ve-^rfeTted from operations. When viewed as an inoti- 
tution, the research f 'inct lon^has also been separated from the work^a-day 
world, flesearch is seen as the province of unlversUles or special institutes 
and think tanks- One result of this separation is that many people see en- 
gaging In research as being at va/rlance with the business of getting things 
done. The researcher or evaluntor U seen as a fbreifrner, who thinks and ^ 
writes a different Language. " - ninne research is outside 'the day-to-day world 
of work, when one enters that world it is often seen as an interruption. In 

fact, in mny ways 'reseeirph is a disruption. 

Decjl^ionfl already made '. yWhen an organization has identified a ^pblem, 
searched for alternative responses, evaluated Its alternatives, and decided , 
to move ahe^d with a particular orp;ttnl7atlon devf^lopment scheme, it has in 
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fact invested heavily in eva|Luation activities.^ Many of W •who are concerned 

with evaluation research 'tead^to overlook this heavy^ early investment by the 

\ ■ . ■ 

client. To later suggest thatHhe project be evaluated might understandably 

appear t©> the client as redundaiit or as second-guessing his decisions. The 

potential value- of evaluation research after a change project is launched,. 

may not be self-evident to the. client. He'may need to be convinced of the. 

utility in. evaluating a- coinmitnient already irad^.^One way to accomplish this 

is through the client !s. lnvc5?Rrqjfient/in designing the process of evaluation 

(Campbelli' l969). - .. , , . . . • . 

Distrust of the a4^uacy ^ attempts at evaluation . All complex organiza 
tions, have elaborate monitoring, evaluation, and u:pntrol systems which attempt 
to measure the quantitative and qualitative ^nature of ongoing activities. 
Management information systemsi, production control system;^, inventory control 
" systems, etc. become more and more sophisticated' each year. However, our 
direct experiences witVv such systems of measurement teach us that often 
these systems do not adequately mirror the ''real'^' state of affairs within 
our organizations.. We see thei^ inadequacies e^en where the a'^tivities being 
measured appear to -be readily quantified. Our experience further tells 'us that 
when we try to measure complex and soft variables — such as how people^ are be- 
haYing— it is difficult to achieve valid results. The client's experience 
therefore leads, to a scepticism (which in many ways is quite healthy) as to 
what -Gisin and cannot be evaluated or assessed. \ 

"Fear of results .*' A fourth obstacle within the client is the apprehension 
most of us have about evaluationT Namely, what is learned may be disturbing 
or embarrassing. It may -call for action he doesn't want to take. The fre- 
quent rei^uctance of mny people to see a physician when they find a lump has 

. . • ■ \ 

developed someplace in their body illustrates thi^ concern. Ir^^ct, this 

• n ' * ' / 

example from the field of public health suggests a 
those organizations which most need the inspectio/i 
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research are those least interested in seeking such infonnation. 

Obstacles in the OD Pra ctitioner ' . ' ' 

5 , • 

. The obstacles found in the.^lient organization are often mirrored within 

"C • • ■ ■ • 

the OD practitioner. The OD prof esf^ional may be apprehensive about the results 
of any evaluation, he may distrust' the^ validity of evaluation, processes , he^ 
may see eyaluation as- disruptive, and h^ may already be cohvinced that the / 
decision to embark on an OD project is right. In- addition, there may be 
' 6ven more obstacles within the professional. Anyone who has been-an active 

producer or a consumer of field research studies knows of the %riad of pro- 

\ ' ^ 
blems, traps, and difficulties in conducting respectable research in ap' organiza- 

\V 

^tional setting. In a. recent survey of members ^^^f the National OD Network, 
Armenakis (1973) reports that the selection and quantitative measurement of \ 
criteria was the problem most frequently mentioned by professionals in this 
field. The second and third ranking problems were difficulties in using com- I 
parison groups and "in controlling for extraneous influences. 

^ The complexities/of these problems can give rise to one or both of .the 
following questions within the OD practitioner: 




(1) Do I (or do available colleagues) have the required 
competence in evaluktion research to conduct -V sounds 
, , evaluation of the OD project given the enormous' 

difficulties? 

. ' (2) .Will the results of an evaluation be suffio^/ently valid 

^(and convincing) to justify the' time and Veaources invol- 
^ ved? Will I 'be able to professionally support the results? 

. * ' . / * • ' * 

Obstacles in the OD Activity Itself • . ' ^ 

There are two obvious obstacles' within the OD activity ^tself. First, 
the method of evaluation or documentation which has been chosen may ^not fit. 
' . - It may n6t prove useful as the project unfolds, Where^evaluatUun is, seen as 
' . an intrusion, or as something which is be3ide-the-pof?it but vill hav« to be 

^ endured, then people are likelj^ to find lots of reasbns for not .getting Invol- 

ved in evaluation: A second factor within the -OU activity is that often such 

ErJc . 14 ' . ' , . , 



activities are seductive and people become ego-involved in ways that^reduce 
their ability to step back and "view the project with dispassi'on and objecti- 
vity. There are sometimes ^moments in OD projects 'which generate affect simi- 
lar to the intense emotional experiences found in. T-^roups- Where such a , 
climate pervades the. OD. proj'ect, .evaluation is easily viewed as irrelevant, 
if not profane. Recently, after a team building session in Chicago with an 
industrial client one of the authors walked more than six miles to cool -him- 
self out (a very uncharacteristic behavior for h/im;. ' 

> Two Issues. to Be Addressed 

* / 

Whenever a cost-benefit analysis yields a "go" signal to evaluate an OD 

change effort, several inrportant Issues remain to be considered no matter 

what mode 1^ of-, evaluation one chooses. We have chosen to address two issues^ 

because they are not treated in the same manner elsewhere artd. because a more 

exhaustive survey of issues is beyond the scope of thlar paper- Two issues * 

which must be fa<jed in any effort at" evaluation are clarity of the r^le of 

evaluation aind the audience to which the evaluation is addressed. 

Clarity of the role of evaluation . Two decisions assist .in clarifying 
i> * • . 

the role of evaluation within the framework of the OD project. First, ''are 

the action. functions to jDe separated from the evaluation research functions, 

in terms of (l) wh^,^g^s eachj^LiC?) are they to be separated in time during 

the project, andy^(3) are they to' be lntegac*ated' together into the fabric of 

the project? As we noted earlier, each of these approaches to evaluation has 

its own advantages and disadvaiitages . Herle we are simply raising the issue 

of. clarity. It is very helpful if there is clarity and srhared agreement early 

in the life of the project about when fiind how evaluation is to, take place. 

Similarly, there must also be clarity about who is to assume responsibility 

for evaluation. Where responsibility is unclear, given the other demands 
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mentioned above, it , is ■eas^^Q.^ evaluation to "fall through a crack" and be 
overlooked by default rather than by decision. 

-Second, closely associated wi£h any discussion of the role of evalua- 
tion, is the question, 'Vhat i© the purpose oT evaluation?" It is not 
enbugi? for one to attempt to e\iraluate an OD project because -one thinks* he 
s'b.ould, or beca,use others appear to expect it. If eui evaluation is ^to be 

rj^fi^(hile, it is well to examine its purposes. What will an evaluartion. per- 
mit us.^^C^ay or do that we could not say or do othervrise? Documentation is „ 

often \ju3ed\^ help others understand what you are doing when you yourself . 
\ ' * 

have little opubt of a project *s present and future VB?lue« Whereas, research 
may raise questions about the. C^alue or usefulness of what you are doing or 
the way you are doing It. -Therefore, questions about the purposes of evalua- 
tion raise their heads. Is the purpose to provide data for "believers?" 
Sometimes this increases their confidence in what they are doing. Is the 
purpbse to attempt to persuade nonbelie vers? If they are skeptics, sometimes 
a* new set of carefully collected infonmtion can be persu^sivef. If they are 
confirmed disbelievers with emotional attachment to the;r posi'.ion, there is 
little chance that an evaluation study will change their vleys . "Fully 35^ 
boasts the promoter; . only 3% slphs the detrocter" (Weiss , 1972a, p. 32). 
Is the purpose to critically exam^rie \Jhat 's happening in the'OD project and ^ 
what its consequences seem to\e'! One must be prepared to face both positive 
and negative outcomes from research of this nature. 

The Audlfence. There are three obvious consumers of the results of • any 
evaluation of an. OD project: ' the client organization, the OD consultant' him- 
self, and e^ctemal audiences, including those who are members of the client 
organization but vho are/external to the OD project. Attention should be 
giverf to the relative iLportance. of each of these three audiences . Several 
decisi^^ are" contingent upd^ determining the audience for the evaluation data; 
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vrhat ^evaliktion modfeL to employe what ^data^to, collect, when the feedback of 

data takes place, and in what form these data are fedback. An evaluation 

study which might be judged to be an outstanding article by editors of the 

Journal of Applied. Behaviorad Science could be worth little as input Into 

the client system--the timing might be too late, the -wording too technical, 

and the tone too/ general and imperscnal/^l Convfers^ly, evuluation resuJ^ts'_ 

which the client may find intriguing and^iiseful might be vijpwed with little 

interest .by outsideirs, because documentation is. seen*^s ina'dequ^te or evl-' 

dence pf^change as ojnbijguo'us. / ^ * . / , — • - 

Of thesje thi*ee potential'^ audiences, the one we ^-sujBpect is most easily 

overload or underwelghted is the OD consultant. DocC&nentatlon and evalua-\ / 

tion of our efforts .'can iserve both to increase our un^rstanding of the com- , 

plex, long-term^" change" activities in which we* become involved, as we^l'aa: 

to enha^fce our skilly in subsequent efforts • Fritz Roethlisbejrger once said 

\hB.t one* problem with managers is that^they Just 'don' t learn from their exper- ^ 

ience. Experience per se in organi^^at ion] development is no g^iarantee that 

we are becoming more professionally competent. Evaluation dat^ can provide 

opportunities to ieam from experience— opportunities that the experiences 

theinselves do not provide. Recently, Cflark stated, "The need for codification 

was driven home to me when, at a conference, I was asked by a >oung graduate 

student in psychology, 'How do you know what to do when you intervene in the , ^ 

life of an organization?' I bluffed, bumbled, and., fumbled. Later I tried 

to analyse why my response was so inadequate. I came up with the. reason that, 

as 3^b, there has been little codification of our experience as practitioners 

and little connection made between theory and practive" (1973, P- 6U0). 

Friedlander and Bfovm (197^) contend that "the mpre sophisticp.ted our efforts 

♦ 

at evaluation and validation become the more lively we will be able to deve- 
lop research methods and competencies ..^ so that our cha'^e efforts will bbcome 

IV 
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more and more research directed and data based, as opposed to being baaed 
on the exploration of good intentions ' , ' 

Finally, evaluation studies can^ncrease the visibility of the OD pro- 
fessional both within and outside his or her organization. Such visibility 
sometimes provides opportunities for opening minds that are skeptical or 
opening doors which otherwise would remain closed; 

.Once evaluation data are collected, our interest is likely to turn to 
the question of how widely this inf ormation ,can be disseminated. Who are the ^ 
appropriate audience^? Every organization has proprietary Interests and some 
* concerns about confidentiality. These matters need to ^e' clarified and under- 
stood by the client and^'the researcher before the study begins.. Both authors 
have been Impressed with the great differences between organizations about 

■ ■■■ - K 

how^open they- are- i^ith da.ta collected internally. Often the policies and 
expectations of an organization cannot be infeirred either from its ccimnit- 
^ment tg ^xperim^nt and change or in the face-to-f^ace behavior of members of 
that organization. 

As ia pre>^ioufl sections of this paper, we have* left hundreds of questions 
unstated.. In addition,*^ we have-not addressed two major issues which are jiure 
to arise whenever a^decision to evaluate an OD project is under consideration: 
(l> relevant 'data— -who provides, the data, who provides the criteria, when are 
data gathered, who cojllects the data, and are explicit linkages made between 
the data collected and the expressed goals of the organization? (2) Appro- 
priat? research procedui'es— what is, the design of the evaluation stud!y, is an 
action^res^arch model to',be emplo^^^d , " i s the study to use' a clinical approach 
or' is it ^o be ba^sed on statistiyi^al analyses, are control or, coniparison groups 
to be .used?* Carol Weiss addresses these issues of relevant 'data and choosing " ^ 
reseekrch jJrocedures in her book Evaluation Research (1972a). 
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\^ Summary 

Models of five alternative .ways^ to approach tfte evaluation of OD pro-* 
Jects were sketched: (l) Ask the expert; (a) What 7s the target?; (3) Did 
vte hit the . tj%rget? ; (h) Mid-course correction; and/ (5) ContinuQos monitor- 
ing. Each model provides a different perspective op the problem of evalua- 
tion and each contains certain advantages and disadvantages. The five 
approaches^ were n^xt put into organizational context by a survey of some ^ 
major obstacles to effective evaluation studi^ . Persistent obstacles were 
found in the client, in- the OD consultant, and within the OD activity 'itself 
In the f ip/al/section, two issues were addressed which must be faced in any 
effort to evaluate an OD program — clarity of the role of evaluation and the 
audience of the evaluation. 
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SYSTEMS EVALUATION CRITERIA, Jo Ann J. Chanoux. , * \ 

THE DESCRIPTI^ VALTDnY OF THE STATIGKARITY ASSUMPTIOlJ IN TIME 
DISCOUNTING:' AN EXPLGRATOEY STUDY, Herbert Moskowitz & John Hu^es. 

A RESOURCE MARKET ENTTrMA IN PRIJJCTPLES COURSES - SOME UNCHARTED 
LINKAGES, Robert V, Hortan. ^ 

PARTIAL POOLING: A HEtSRISTIC, Dick R. Wittlnk. 

AN EMPIRICAL-SIMUIATION 1 APEROI^^ TO COMPETITION, Randall L. Schiatz 
and Joe>;A. Dodson, Jr. ^ 

EROTIC MATERIALS : A CdtfMMODITY THEORY ANALYSIS THE ENHANCED 
DESIRABILITY WHICH MAY^ ACCOMPANY THDEm UNAVAILABILITY, Howard L. 
Fromkln emd Timothy c/ Brock. 

MJLTIFIRM ANALYSIS OF COMPETITIVE DECISION VARIABLES, Albert R. 
Wildt and Frank M. Bass. 

EARNINGS VARIABILiry AS A RISK SURROGATE, Ruflsell M. Baref ield 
and Eugene E. Ccxmlskey. 

MARKET STRUCTURE AND HtOFIIABILITY - ANAUSIS OF Hffi AJTBOERIATE- 
KESS OF JOOUm CROSS-SECTICaSAL INDUSTRY DATA, Frank M. Baas. 

■BHE EXHANATORT. EFFICACnr OF SEIBCTED TYPES OF CONSUMER IKCFILE 
VARIABIES IN TASHiaN CHANGE AGENT IDENTIFICATiaN, Charles W. 
King and GeeMfge B. Sproles. 

cilOUP DECISION-MAEOUG PERFOEMAHCE AS 3JIFLUENCED BY CONSENSUS ANd' 
SEIT-CRIEKTATiaN, Paul M. Nemlroff . 

AN AIGORITHM TOR DETERMlinNG BAYESIAN ATTRIBUTE SINGLE SAMPLING 
ACCEPTANCE PLANS, Herbert Moskovrltz. ' ' - 
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SOME OBSERVATiaNS ON THEORIES OF COLIECTlVE DECISIONS, Herbert J 
MoskavriLtz. , " 

CENTRAIJZATIQN VERSUS 'DEC53miALI2J^ VIA REPORTS OF E5CCEPTI0NS: 
DESCRIPTIVE VERSUS NORMATIVE BEHAVIOR IN A SIMULATED FINAirciAI. 
ORGANIZATION, Herbert Moskowitz and W. Murnighan* - 

EXPERIMENTAL STUDIES OF DISCRIMINATION IN THE EVALUATION OF JOB 
APPLICANTS' RESUMES: I. REIATtVE IMPORTANCE OF SEX, ATfRACTIVElfESS, 
AND SCHOIASTIC STAOTSjlKl}. - y • 

STRATEGIC RESPONSES TO TECHNOLOGICAL HIREATS, A. ibooper, 'E. Denittzzio, 
K., HattMi, E. Hiclss and t). Tock. 

COMMEIRCIAL LOANS AND DEPOSITS OF' LARGE COMMERCIAL BANKS, Robert 0. 
Bdmister. i 

THE CONSUMER FASHION CHANGE- AGENT: A THEORETICAL C0NCEPTUALI2iATI0N' 
AND EMPIRICAL IDENTiriCATICIN,- Geprge B. .Sprolea and Charles W. King". 

RESEARCH PERSPECTIVES FOR iviATERIAL REQUIREMENTS PIAHNIHG SYSTEM©,. 



W. L. Berry and D. Clay ^hybark. 



1+35 JOINT-SPACE ANAIYSIS OF THE STRUCTURE OF AFFECT USING SINGLE-SUBJECT 
' DISCEIMINAHT CONFIGURATIONS: PART-1, E. A. ^eBsemier 

1+36 IMPLEMENTATION. AND ORGANIZATIONAL ^ltALIDITI : AN EMPIRICAL INVBSTIOATIOW, 
Randall L: SchUltz dnd Dennis P. Slevin. 

kyj OPTIMAL AUDIT. PLANNING - PART 1. Jahn S. Hughes. 
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